AutoMap  Component 
Reference  Manual 


Michael  W.  Bigrigg 

August  2009 
CMU-ISR-09-124 


Institute  for  Software  Research 
School  of  Computer  Science 
Carnegie  Mellon  University 
Pittsburgh,  PA  15213 


Center  for  the  Computational  Analysis  of  Social  and  Organizational  Systems 
CASOS  technical  report. 


This  work  was  supported  in  part  by  the  Office  of  Naval  Research  under  Contract  No.  N00014081 1223  to 
do  the  componentization  of  AutoMap,  and  Contract  No.  N00014081 1 186  for  work  on  thesauri,  and 
Contract  No.  N00014081 1 186  for  extensions  to  handle  beliefs  and  attributes;  by  the  Army  Research 
Institute  under  Grant  No.  W91WAW07C0063  to  handle  the  part  of  speech  analysis  and  anaphora 
resolution;  and  by  the  Air  Force  Office  of  Scientific  Research  under  Contract  No.  FA9550-05-1-0388. 
Additional  support  was  provided  by  the  Center  for  Computational  Analysis  of  Social  and  Organizational 
Systems  (CASOS)  at  Carnegie  Mellon  University.  The  views  and  conclusions  contained  in  this  document 
are  those  of  the  author  and  should  not  be  inteipreted  as  representing  the  official  policies,  either  expressed 
or  implied,  of  the  Office  of  Naval  Research,  the  Air  Force  Office  of  Scientific  Research,  the  Army 
Research  Institute  or  the  U.S. 


Report  Documentation  Page 

Form  Approved 

OMB  No.  0704-0188 

Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 

VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  OMB  control  number. 

1.  REPORT  DATE 

AUG  2009  2.  REPORT  TYPE 

3.  DATES  COVERED 

00-00-2009  to  00-00-2009 

4.  TITLE  AND  SUBTITLE 

AutoMap  Component  Reference  Manual 

5a.  CONTRACT  NUMBER 

5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

6.  AUTHOR(S) 

5d.  PROJECT  NUMBER 

5e.  TASK  NUMBER 

5f.  WORK  UNIT  NUMBER 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

Carnegie  Mellon  University  , School  of  Computer 

Science, Pittsburgh, PA, 15213 

8.  PERFORMING  ORGANIZATION 

REPORT  NUMBER 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

10.  SPONSOR/MONITOR'S  ACRONYM(S) 

11.  SPONSOR/MONITOR'S  REPORT 
NUMBER(S) 

12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release;  distribution  unlimited 

13.  SUPPLEMENTARY  NOTES 

14.  ABSTRACT 

15.  SUBJECT  TERMS 

16.  SECURITY  CLASSIFICATION  OF:  17.  LIMITATION  OF 

_ _ _  ABSTRACT 

18.  NUMBER  19a.  NAME  OF 

OF  PAGES  RESPONSIBLE  PERSON 

a.  REPORT  b.  ABSTRACT  c.  THIS  PAGE  Same  OS 

unclassified  unclassified  unclassified  Report  (SAR) 

73 

Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std  Z39-18 


Key  Words:  Semantic  Network  Analysis,  Dynamic  Network  Analysis,  Component 
Programming,  Social  Networks,  AutoMap 


Abstract 


AutoMap  is  software  for  computer-assisted  Network  Text  Analysis  (NT A).  The  AutoMap 
components  are  a  collection  of  programs  that  implement  the  features  available  in  AutoMap. 
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AutoMap  Components 

The  AutoMap  components  are  a  collection  of  over  60  java  executable 
programs  for  use  in  Network  Text  Mining  applications.  These  components 
are  used  by  interface  programs  such  as  AutoMap  GUI,  AutoMap  Script,  and 
SORASCS. 

How  To  Use 

The  components  require  Java  1.6  or  higher  and  are  callable  via  a  command 
line  interface.  The  following  example  shows  how  to  call  the  Lowercase 
routine. 

C:\>  java  -cp  am3.jar  edu. emu. casos. automap. Lowercase  C:\in  C:\out 

The  C:\in  and  the  C:\out  directories  are  example  input  and  output 
directories  and  are  given  as  command  line  parameters  passed  to  the 
Lowercase  routine. 

The  AutoMap  components  are  not  directly  callable  from  a  Java  program 
except  to  the  extent  to  which  any  main()  may  be  called.  No  guarantee  is 
given  to  components  being  called  in  this  manner. 

Classifications 

Preprocessing  Filter 
Generation  Routine 
Supplemental  Procedure 
File  Utility 
CEMap  Component 
Internal  Command 
External  Tool 

Component  Reference 

This  section  gives  an  alphabetical  list  of  all  AutoMap  components.  This  list 
may  not  include  routines  that  are  used  by  the  AutoMap  components 
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indirectly  or  routines  that  may  be  called  by  3rd  party  libraries  that  AutoMap 
may  make  use  of. 


Mandatory  arguments  are  listed  in  <angle  brackets>  with  the  pipe  |  symbol 
being  used  to  identify  accepted  options.  Optional  arguments  are  in  [square 
brackets].  Routines  marked  with  an  asterisk  (*)  denote  components  that 
heavily  leverage  a  3rd  party  library,  other  than  libraries  provided  with  Java. 
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Add  Attribute 


Classification:  Supplemental  Procedure 

Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. AddAttribute  <semanticnet_dir> 
<output_dir>  <attrib.csv> 

Parameters: 

<semanticnet_dir>  input  directory  of  DyNetML  files 
<output_dir>  output  directory  of  DyNetML  files 

<attrib.csv> 
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Add  A  ttribute3Col 


Classification:  Supplemental  Procedure 

Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. AddAttribute3Col 
<semanticnet_dir>  <output_dir>  <attrib.csv> 

Parameters: 

<semanticnet_dir> 

<output_dir> 

<attrib.csv> 
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Addlmages 


Classification:  Supplemental  Procedure 

Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. Addlmages  <network  file > 
<outfile>  [-k] 

Parameters: 

<network  file > 

<outfile> 

-k  flag  preserves  pre-existing  images 
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Add  Meta 


Classification:  Internal  Command 

Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. AddMeta  <metadata_file> 
<input_dir>  <output_dir>  <column_type> 

Parameters: 

<metadata_file> 

<input_dir> 

<output_dir> 

<column_type> 


6 


AddTimeFUeName 


??? 

Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. AddTimeFileName  <input_dir> 

Parameters: 

<input_dir> 
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Anaphora 


???? 

Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. Anaphora  <input_dir> 
<output_dir> 

Parameters: 

<input_dir> 

<output_dir> 
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AttributesToProps 


Classification:  Supplemental  Procedure 

Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. AttributesToProps  <input_dir> 
<output_dir> 

Parameters: 


<input_dir> 

<output_dir> 
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Belief  Enhancement 


Classification:  Supplemental  Procedure 

Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. BeliefEnhancement  <input_dir> 
<output_dir>  <belief_enhancement_file>  <network  type  m|s> 

Parameters: 

<input_dir> 

<output_dir> 

<belief_enhancement_file> 

< network  type  m|s> 
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BigramText 


Classification:  Internal  Command 

Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. BigramText  <input_dir> 
<output_dir> 

Parameters: 


<input_dir> 

<output_dir> 
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CRFSuggestion 


Classification:  Generation  Routine 

Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. CRFSuggestion  <input_dir> 
<output_dir>  <crf_dir> 

Parameters: 

<input_dir> 

<output_dir> 

<crf  dir> 
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CharsetToolkit 


Classification:  Preprocessing  Filter 

Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. CharsetToolkit  <input> 
<output_dir>  [Directory |  perFile] 

Parameters: 

<input> 

<output_dir> 

[Directory  |  perFile] 

Note:  input  can  either  be  a  directory  or  a  file 
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Clicklt 


Classification:  Supplemental  Procedure 

Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. Clicklt  <network_file> 
<output_file>  <location> 

Parameters: 

<network_file> 

<output_file> 

<  location  > 
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CombineThesa  uri 


Classification:  Supplemental  Procedure 

Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. CombineThesauri  <output_file> 
[thes_file]  [thes_file]  ... 

Parameters: 

<output_file> 

[thes_file] 
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ConceptList 


Classification:  Generation  Routine 

Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. ConceptList  <input_dir> 
<output_dir>  <gram_type> 

Parameters: 

<input_dir> 

<output_dir> 

<gram_type> 
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Con  vertTextEncoding 
Classification:  Preprocessing  Filter 
Classification:  Preprocessing  Filter 

Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. ConvertTextEncoding  <input> 
<output_dir>  <encoding>  [perDirectoryl perFile] 

Parameters: 

<input> 

<output_dir> 

<encoding> 

[perDirectoryl  perFile] 

Note:  input  can  either  be  a  directory  or  a  file 
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Dedupe 

Classification:  File  Utility 
Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. Dedupe  <input_dir>  <output_dir> 
<duplicate_dir>  <log_dir> 

Parameters: 

<input_dir> 

<output_dir> 

<duplicate_dir> 

<log_dir> 
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Delete 


Classification:  Preprocessing  Filter 

Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. Delete  <input_dir>  <output_dir> 
<D|R>  <delete_file> 

Parameters: 

<input_dir> 

<output_dir> 

<D|R> 

<delete  file> 
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DyNetMIEditor 
Classification:  External  Tool 
Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. DyNetMIEditor  <input_file> 
<output_file>  < node_list _ file> 

Parameters: 

<  i  n  p  u  t_f  i  I  e  > 

<output_file> 

<node  list  file > 
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ExtractNumerics 


Classification:  Generation  Routine 

Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. ExtractNumerics  <input_dir> 
<output_dir> 

Parameters: 


<input_dir> 

<output_dir> 
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FetchMail 


Classification:  CEMap  Component 

Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. FetchMail  <protocol> 
<username>  <password>  <server>  [folder]  <output_body> 
<output_header> 

Parameters: 

< protocol > 

<username> 

<  password  > 

<server> 

[folder] 

<output_body> 

<output_header> 

protocol  may  be  any  of  imap,  imaps,  pop3,  or  pop3s 
server  string  is  of  the  form  address[:port] 
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FilterDirectory 
Classification:  File  Utility 
Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. FilterDirectory  <input_dir> 
<output_dir>  <filter> 

Parameters: 

<input_dir> 

<output_dir> 

<filter> 
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FilterText 


Classification:  File  Utility 
Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. FilterText  <input_dir> 
<output_dir> 

Parameters: 

<input_dir> 

<output_dir> 
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General 


Classification:  Preprocessing  Filter 

Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. General  <input_dir> 
<output_dir>  <thesaurus> 

Parameters: 


<input_dir> 

<output_dir> 

<thesaurus> 
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GenerateMetaNetwork 


Classification:  Generation  Routine 

Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. GenerateMetaNetwork 
<metalist_dir>  <semanticlist_dir>  <properties_dir>  <output_dir> 

Parameters: 

<metalist_dir> 

<semanticlist_dir> 

<properties_dir> 

<output_dir> 
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GenerateSemanticNetwork 


Classification:  Generation  Routine 

Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. GenerateSemanticNetwork 
<concept_dir>  <semanticlist_dir>  <properties_dir>  <output_dir> 

Parameters: 

<concept_dir> 

<semanticlist_dir> 

<properties_dir> 

<output_dir> 
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KStemMain 


Classification:  Preprocessing  Filter 

Usage: 

java  -cp  am3.jar  edu.cmu.casos.aut 
<output_dir> 


Parameters: 


<input_dir> 

<output_dir> 


.KStemMain  <input_dir> 
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KStemmer 


Classification:  Preprocessing  Filter 

Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. KStemmer  <input_dir> 
<output_dir>  <stem_capitalization  y|n> 

Parameters: 

<input_dir> 

<output_dir> 

<stem_capitalization  y|n> 
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Keep 


Classification:  Internal  Command 

Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. Keep  <keep_file>  <input_dir> 
<output_dir>  <adjacency> 

Parameters: 


<keep_file> 

<input_dir> 

<output_dir> 

<adjacency> 
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KeepConcepts 


Classification:  Internal  Command 

Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. KeepConcepts  <input_dir> 
<output_dir>  <keep_file> 

Parameters: 


<input_dir> 

<output_dir> 

<keep_file> 
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KeyWordlnContext 
Classification:  Generation  Routine 


Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. KeyWordlnContext  <input_dir> 
<output_dir> 

Parameters: 

<input_dir> 

<output_dir> 
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Lowercase 


Classification:  Preprocessing  Filter 

Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. Lowercase  <input_dir> 
<output_dir> 

Parameters: 


<input_dir> 

<output_dir> 
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Lo  wercaseConcepts 
Classification:  Internal  Command 
Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. LowercaseConcepts  <input_dir> 
<output_dir> 

Parameters: 


<input_dir> 

<output_dir> 
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Merge 

Classification:  Supplemental  Procedure 

Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. Merge  <input_dirl>  <input_dir2> 
<output_dir> 

Parameters: 


<input_dirl> 

<input_dir2> 

<output_dir> 
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MergeDeleteLists 

Classification:  Supplemental  Procedure 

Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. MergeDeleteLists  <output_file> 
[delete_list_file]  [delete _list_file]  ... 

Parameters: 

<output_file> 

[delete_list_file] 
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MetaNet 


Classification:  Generation  Routine 

Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. MetaNet  <concept_dir> 
<semantic_dir>  <properties_input_dir>  <output_dir> 

Parameters: 

<concept_dir> 

<semantic_dir> 

<properties_input_dir> 

<output_dir> 
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MetaNetText 


Classification:  Generation  Routine 

Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. MetaNetText  <input_dir> 
<output_dir>  <metanet_thesauri> 

Parameters: 

<input_dir> 

<output_dir> 

<metanet  thesauri> 
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NamedEntity 

Classification:  Generation  Routine 

Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. NamedEntity  <input_dir> 
<output_dir> 

Parameters: 

<input_dir> 

<output_dir> 
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PStemmerMain 


Classification:  Preprocessing  Filter 

Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. PStemmerMain  <in 
<output_dir>  <porter_language> 

Parameters: 

<input_dir> 

<output_dir> 

<porter_language> 


put_dir> 
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PdfConverter 


Classification:  File  Utility 
Usage: 

java  -cp  am3.jar  edu.cmu.casos. 
<output_dir> 

Parameters: 


<input_dir> 

<output_dir> 


.PdfConverter  <input_dir> 
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PosTagger 

Classification:  Generation  Routine 

Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. PosTagger  <input_dir> 
<output_dir>  <POS_dir>  <posType  aggregatel ptb>  <save_output_as 
csv|txt> 

Parameters: 

<input_dir> 

<output_dir> 

<POS_dir> 

<posType  aggregatel ptb> 

<save_output_as  csv|txt> 
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PositiveThesa  uri 


Classification:  Generation  Routine 

Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. PositiveThesauri  <input_dir> 
<output_dir> 

Parameters: 


<input_dir> 

<output_dir> 
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Properties 

Classification:  Generation  Routine 

Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. Properties  <input_dir> 
<output_dir> 

Parameters: 

<input_dir> 

<output_dir> 
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RemoveExtra  White 


Classification:  Preprocessing  Filter 

Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. RemoveExtraWhite  [-t]  [-n] 
<input_dir>  <output_dir> 

Parameters: 

<input_dir> 

<output_dir> 

-t  flag  causes  excess  tabs  to  be  removed 
-n  flag  causes  excess  newlines  to  be  removed 
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RemoveNumbers 


Classification:  Preprocessing  Filter 

Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. RemoveNumbers  <input_dir> 
<output_dir> 

Parameters: 


<input_dir> 

<output_dir> 
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RemovePunct 


Classification:  Preprocessing  Filter 

Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. RemovePunct  <input_dir> 
<output_dir> 

Parameters: 


<input_dir> 

<output_dir> 
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RemoveSym 


Classification:  Preprocessing  Filter 

Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. RemoveSym  <input_dir> 
<output_dir> 

Parameters: 


<input_dir> 

<output_dir> 
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RemoveUserSym 
Classification:  Preprocessing  Filter 

Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. RemoveUserSym  <input_dir> 
<output_dir>  <symbol_file> 

Parameters: 

<input_dir> 

<output_dir> 

<symbol_file> 
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SemanticList 


Classification:  Generation  Routine 

Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. SemanticList  <input_dir> 
<output_dir>  <window_size>  <direction  B|U>  <window_reset> 

Parameters: 

<input_dir> 

<output_dir> 

<window_size> 

<direction  B|U> 

<window_reset> 

window_reset  should  be  A,  W=n,  S=n,  or  P=n,  n  a  positive  integer 
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SemanticNet 


Classification:  Generation  Routine 

Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. SemanticNet  <concept_dir> 
<semantic_dir>  <properties_dir>  <output_dir> 

Parameters: 

<concept_dir> 

<semantic_dir> 

<properties_dir> 

<output_dir> 
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SpiderDriver 

*** 

Classification:  CEMap  Component 

Usage:  java  -cp  am3.jar  edu. emu. casos. automap. SpiderDriver  [options] 
<output_dir>  <URL> 

Parameters: 

<output_dir> 

<URL> 

[options] 

--documents 
--max-size=n 
— quota=size 
-random-wait 
— timeout=n 
— tries=n 

— user-agent=string 
—wait=n 

All  numeric  values  must  be  nonnegative,  a  limit  of  0  means  no  limit 
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SubmatrixSelection 


Classification:  Internal  Command 

Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. SubmatrixSelection 
<metajist_dir>  <types_file>  <output_dir> 

Parameters: 

<meta_list_dir> 

<types_file> 

<output_dir> 
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ThesauriSort 


Classification:  Supplemental  Procedure 

Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. ThesauriSort  <thesaurus_file> 
<output_file> 

Parameters: 

<thesaurus_file> 

<output_file> 
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Threshold 


Classification:  Internal  Command 

Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. Threshold  <input_dir> 
<output_dir>  <threshold> 

Parameters: 


<input_dir> 

<output_dir> 

<threshold> 
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Union 


Classification:  Supplemental  Procedure 

Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. Union  <input_dir>  <output_dir> 
<union_file_name>  <union_type> 

Parameters: 

<input_dir> 

<output_dir> 

<union_file_name> 

<union_type> 
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UnionConcepts 

Classification:  Supplemental  Procedure 

Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. UnionConcepts  <input_dir> 
<output_dir> 

Parameters: 


<input_dir> 

<output_dir> 
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Union  KeyWordsInContext 
Classification:  Supplemental  Procedure 

Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. UnionKeyWordsInContext 
<input_dir>  <output_dir> 

Parameters: 

<input_dir> 

<output_dir> 
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Uppercase 


Classification:  Preprocessing  Filter 

Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. Uppercase  <input_dir> 
<output_dir> 

Parameters: 


<input_dir> 

<output_dir> 
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UppercaseConcepts 

Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. UppercaseConcepts  <input_dir> 
<output_dir> 

Parameters: 

<input_dir> 

<output_dir> 
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WhiteNumbers 


Classification:  Preprocessing  Filter 

Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. WhiteNumbers  <input_dir> 
<output_dir> 

Parameters: 


<input_dir> 

<output_dir> 


61 


WhitePunct 


Classification:  Preprocessing  Filter 

Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. WhitePunct  <input_dir> 
<output_dir> 

Parameters: 


<input_dir> 

<output_dir> 
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WhiteSym 

Classification:  Preprocessing  Filter 

Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. WhiteSym  <input_dir> 
<output_dir> 

Parameters: 

<input_dir> 

<output_dir> 
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WordDocCon  verier 


Classification:  File  Utility 
Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. WordDocConverter  <input_dir> 
<output_dir> 

<input_dir>  <output_dir> 
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WordList 


Classification:  Internal  Routine 

Usage: 

java  -cp  am3.jar  edu. emu. casos. automap. WordList  <input_dir> 
<output_dir> 

Parameters: 


<input_dir> 

<output_dir> 
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